Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore!: improve ChunkProofVerification #2446

Merged
merged 4 commits into from
Nov 26, 2024

Conversation

maqi
Copy link
Member

@maqi maqi commented Nov 15, 2024

Description

🚨 Breaking Changes

improve ChunkProofVerification

sn_node/src/node.rs Fixed Show fixed Hide fixed
sn_node/src/node.rs Fixed Show fixed Hide fixed
@maqi maqi force-pushed the improve_chunk_proof_verification branch 2 times, most recently from 2f72a3d to 1890cf5 Compare November 18, 2024 12:27
@maqi maqi changed the title chore: improve ChunkProofVerification chore!: improve ChunkProofVerification Nov 18, 2024
sn_node/src/node.rs Fixed Show fixed Hide fixed
sn_node/src/node.rs Fixed Show fixed Hide fixed
sn_node/src/node.rs Fixed Show fixed Hide fixed
Copy link
Member

@grumbach grumbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work in general

sn_node/src/node.rs Outdated Show resolved Hide resolved
sn_node/src/node.rs Outdated Show resolved Hide resolved
let mut all_chunk_addrs: Vec<_> = all_local_records
.iter()
.filter_map(|(addr, record_type)| {
if *record_type == RecordType::Chunk {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only chunks? I think all records should be checked.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

register/spend has nature of could have different versions under the same name,
which, it's not guaranteed to be always consistence across the network.
they are also small in generally cases, which cheaper to be cached, and include them could pollute the scoring ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They can be self verifiable, so even if different, the sig is what matters

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we only verify storage of chunks what's my incentive as a node to keep other records?
We should check all records regardless of what type.

Copy link
Member Author

@maqi maqi Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They can be self verifiable, so even if different, the sig is what matters

Unfortunately, they are not.
To verify the signature, the owner's pub key must be known.
But nodes doesn't have that info, unless being contacted by the client.

suggested to have client check all types of records

sn_node/src/node.rs Outdated Show resolved Hide resolved
@maqi maqi force-pushed the improve_chunk_proof_verification branch from 1890cf5 to 3dfd466 Compare November 19, 2024 11:16
sn_node/src/node.rs Fixed Show fixed Hide fixed
sn_node/src/node.rs Fixed Show fixed Hide fixed
@maqi maqi force-pushed the improve_chunk_proof_verification branch from 3dfd466 to 1eb10ae Compare November 20, 2024 13:18
// Sort by distance and only take first X closest entries
all_chunk_addrs.sort_by_key(|addr| key.distance(addr));

// TODO: this shall be deduced from resource usage dynamically

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note

Suspicious comment

let index: usize = OsRng.gen_range(0..num_of_targets);
let target = verify_candidates[index].clone();
// TODO: workload shall be dynamically deduced from resource usage

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note

Suspicious comment
Ok((peer_id, score)) => {
if score < MIN_ACCEPTABLE_HEALTHY_SCORE {
info!("Peer {peer_id:?} failed storage challenge with low score {score}/{MIN_ACCEPTABLE_HEALTHY_SCORE}.");
// TODO: shall the challenge failure immediately triggers the node to be removed?

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note

Suspicious comment
}
// TODO: For those answers not among the expected_proofs,

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note

Suspicious comment
Copy link
Member

@grumbach grumbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work! I think we should include all records regardless of the type in the check. During verification, differing records can be dealt with by the verifier (fetch unknown reg, spend, scratch). All data is self verifiable so detecting bad data is easy.

sn_node/src/node.rs Outdated Show resolved Hide resolved
sn_node/src/node.rs Outdated Show resolved Hide resolved
let mut all_chunk_addrs: Vec<_> = all_local_records
.iter()
.filter_map(|(addr, record_type)| {
if *record_type == RecordType::Chunk {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They can be self verifiable, so even if different, the sig is what matters

let mut all_chunk_addrs: Vec<_> = all_local_records
.iter()
.filter_map(|(addr, record_type)| {
if *record_type == RecordType::Chunk {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we only verify storage of chunks what's my incentive as a node to keep other records?
We should check all records regardless of what type.

}

pub fn historical_verify(&self, _other: &Self) -> bool {
// TODO: Shall be refactored once new quote filtering scheme deployed

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note

Suspicious comment
// old_quote.quoting_metrics.close_records_stored
// );

// // TODO: Double check if this applies, as this will prevent a node restart with same ID

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note

Suspicious comment
@maqi maqi force-pushed the improve_chunk_proof_verification branch from 9386241 to 9a922db Compare November 22, 2024 14:18
@maqi maqi force-pushed the improve_chunk_proof_verification branch from 9a922db to 9c9f64f Compare November 25, 2024 10:24
@maqi maqi force-pushed the improve_chunk_proof_verification branch from 9c9f64f to 57eb2c3 Compare November 25, 2024 10:25
@maqi maqi removed the DoNotMerge label Nov 25, 2024
Copy link
Member

@grumbach grumbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work Qi!

@maqi maqi added this pull request to the merge queue Nov 26, 2024
Merged via the queue into maidsafe:main with commit 11b659b Nov 26, 2024
29 checks passed
@maqi maqi deleted the improve_chunk_proof_verification branch November 26, 2024 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants